fix(engine): resolve ORDER BY <ordinal> to the Nth output column (P16) - #59
Merged
Conversation
`ORDER BY 2` returned rows in natural insertion order, with no error. The
integer was not being ignored — `OrderByAliasTransformer` promotes any ORDER BY
item that is not a visible column into a hidden SELECT item so it survives
projection, and `NumberLiteral("2")` qualified. So the query sorted, correctly,
on a hidden column whose value is the constant 2, where every row compares equal.
Row count gives no hint of this: `ORDER BY 2 DESC LIMIT 3` returns three rows
that look plausible but are the first three in file order.
The ordinal is positional against the *output* columns, so it is resolved in
`query_engine::apply_multi_order_by_with_context` rather than in the
transformer. The transformer knows the select list but not the output: `SELECT *`
is still unexpanded there and GROUP BY has not run, and both are shapes where
`ORDER BY 2` has to keep meaning the same thing. The transformer's only job is
to stop promoting numeric literals.
Rules pinned against DuckDB before implementing, not assumed:
- `ORDER BY 2` — 2nd output column, under an explicit select list, `SELECT *`,
or after GROUP BY.
- `ORDER BY 1+1` — NOT an ordinal. It is an ordinary constant expression and
sorts nothing, in both engines.
- `ORDER BY 0` / `-1` / out of range — error.
- `ORDER BY 1.5` — error. This is why the transformer skips every numeric
literal rather than only integer-valued ones: promoting `1.5` would leave it a
silent no-op, and only the engine knows the valid range to report. Same
principle as P13 stage 1 — a refusal beats a different query that succeeds.
Columns promoted for ORDER BY visibility (and HAVING's `__hidden_agg_` columns)
are appended after the real output, so they are excluded from the ordinal range:
`SELECT a, b FROM t ORDER BY c, 3` errors rather than landing on
`__hidden_orderby_1`.
Parity 134 -> 139 AGREE (+2 fixed, +3 new coverage, +1 new BOTH_ERR).
`order_by_ordinal_star` and `order_by_ordinal_group_by` were added because the
defect is about output columns, so each construct that changes what those are is
a separate risk — and "top N by total" is where silently returning group order
was most likely to be believed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TmQRCdZUn3RYqyVFoeKRsY
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes P16. Parity 134 → 139 AGREE (+2 fixed, +3 new coverage, +1 new BOTH_ERR); contract holds at 168 cases.
The defect
ORDER BY 2returned rows in natural insertion order, with no error.The integer was not being ignored.
OrderByAliasTransformerpromotes any ORDER BY item that isn't a visible column into a hidden SELECT item, so the key survives projection — andNumberLiteral("2")qualified. The query then sorted, correctly, on a hidden column whose value is the constant2, where every row compares equal.Row count gives no hint:
ORDER BY 2 DESC LIMIT 3returns three rows that look right but are the first three in file order.Where the fix goes, and why
The ordinal is positional against the output columns, so it is resolved in
query_engine::apply_multi_order_by_with_context, not in the transformer. The transformer knows the select list but not the output —SELECT *is still unexpanded there, and GROUP BY hasn't run — and both are shapes whereORDER BY 2must keep meaning the same thing. The transformer's only job is to stop promoting numeric literals.Columns promoted for ORDER BY visibility (and HAVING's
__hidden_agg_) are appended after the real output, so they're excluded from the ordinal range:SELECT a, b FROM t ORDER BY c, 3errors rather than landing on__hidden_orderby_1.Rules, pinned against DuckDB before implementing
ORDER BY 2SELECT *, or after GROUP BYORDER BY 1+1ORDER BY 0/-1/3of 2should be between 1 and NORDER BY 1.5That last row is why the transformer skips every numeric literal, not just integer-valued ones: promoting
1.5would leave it a silent no-op, and only the engine knows the valid range to report. Same principle as P13 stage 1 — a refusal beats a different query that succeeds.Coverage
Four new corpus cases beyond the two that were pinned. The defect is about output columns, so each construct that changes what those are is a separate risk —
order_by_ordinal_group_bymatters most in practice, since "top N by total" is the shape where silently returning group order was most likely to be believed.Plus 3 transformer unit tests and 6 engine integration tests, including one asserting hidden promoted columns stay out of the ordinal range.
Verification
139 AGREE / 13 DIFFER / 14 GAP / 2 BOTH_ERR, contract holds (168 cases)cargo test: 734 + 461 + 1 passed, 0 failedcargo fmt,cargo clippy(7 pre-existing errors onmain, unchanged).exe-suffix noise, same set as the recorded baseline🤖 Generated with Claude Code
https://claude.ai/code/session_01TmQRCdZUn3RYqyVFoeKRsY